[Backport 2.x] Compress and cache cluster state during validate join request (#7321) by andrross · Pull Request #7945 · opensearch-project/OpenSearch

andrross · 2023-06-06T22:59:52Z

Backports #7321 to 2.x

github-actions · 2023-06-06T23:10:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/16987/
CommitID: 91ee919
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-06T23:52:40Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString

URL: https://build.ci.opensearch.org/job/gradle-check/16989/
CommitID: bcde254
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2023-06-06T23:57:11Z

Codecov Report

Merging #7945 (fb56099) into 2.x (efd12df) will decrease coverage by 0.07%.
The diff coverage is 87.93%.

❗ Current head fb56099 differs from pull request most recent head daeaa52. Consider uploading reports for the commit daeaa52 to get more accurate results

@@             Coverage Diff              @@
##                2.x    #7945      +/-   ##
============================================
- Coverage     70.64%   70.58%   -0.07%     
+ Complexity    56851    56780      -71     
============================================
  Files          4703     4704       +1     
  Lines        269104   269091      -13     
  Branches      39774    39771       -3     
============================================
- Hits         190118   189932     -186     
- Misses        62702    62833     +131     
- Partials      16284    16326      +42

Impacted Files	Coverage Δ
...g/opensearch/cluster/coordination/Coordinator.java	`78.89% <ø> (+3.11%)`	⬆️
...ch/cluster/coordination/CompressedStreamUtils.java	`80.00% <80.00%> (ø)`
...rg/opensearch/cluster/coordination/JoinHelper.java	`84.72% <89.47%> (-0.84%)`	⬇️
...ster/coordination/PublicationTransportHandler.java	`85.18% <100.00%> (-0.68%)`	⬇️

... and 452 files with indirect coverage changes

github-actions · 2023-06-08T18:25:29Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17155/
CommitID: 91f9db9
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

kotwanikunal · 2023-06-13T15:54:03Z

@andrross Can you please rebase and iterate to green?

server/src/main/java/org/opensearch/cluster/coordination/JoinHelper.java

github-actions · 2023-06-15T22:20:03Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17726/
CommitID: c785e9f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-16T00:39:59Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testDropPrimaryDuringReplication

URL: https://build.ci.opensearch.org/job/gradle-check/17734/
CommitID: c785e9f
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions · 2023-06-17T00:13:56Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testPressureServiceStats
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testDropPrimaryDuringReplication

URL: https://build.ci.opensearch.org/job/gradle-check/17830/
CommitID: fb56099
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <andrross@amazon.com>

github-actions · 2023-06-21T15:56:54Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/18092/
CommitID: 0dfe96e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-21T23:49:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/18130/
CommitID: 0dfe96e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

andrross · 2023-06-22T16:35:51Z

Flaky test: #7592

…arch-project#7321) * Compress and cache cluster state during validate join request Signed-off-by: Aman Khare <amkhar@amazon.com> * Add changelog and license Signed-off-by: Aman Khare <amkhar@amazon.com> * Add javadoc and correct styling Signed-off-by: Aman Khare <amkhar@amazon.com> * Add new handler for sending compressed cluster state in validate join flow and refactor code Signed-off-by: Aman Khare <amkhar@amazon.com> * Refactor util method Signed-off-by: Aman Khare <amkhar@amazon.com> * optimize imports Signed-off-by: Aman Khare <amkhar@amazon.com> * Use cluster state version based cache instead of time based cache Signed-off-by: Aman Khare <amkhar@amazon.com> * style fix Signed-off-by: Aman Khare <amkhar@amazon.com> * fix styling 2 Signed-off-by: Aman Khare <amkhar@amazon.com> * Use concurrent hashmap instead of cache, add UT class for ClusterStateUtils Signed-off-by: Aman Khare <amkhar@amazon.com> * style fix Signed-off-by: Aman Khare <amkhar@amazon.com> * Use AtomicReference instead of ConcurrentHashMap Signed-off-by: Aman Khare <amkhar@amazon.com> * Use method overloading to simplify the caller code Signed-off-by: Aman Khare <amkhar@amazon.com> * Resolve conflicts Signed-off-by: Aman Khare <amkhar@amazon.com> * Change code structure to separate the flow for JoinHelper and PublicationTransportHelper Signed-off-by: Aman Khare <amkhar@amazon.com> * Remove unnecessary input.setVersion line Co-authored-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Aman Khare <85096200+amkhar@users.noreply.github.com> --------- Signed-off-by: Aman Khare <amkhar@amazon.com> Signed-off-by: Aman Khare <85096200+amkhar@users.noreply.github.com> Co-authored-by: Aman Khare <amkhar@amazon.com> Co-authored-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit b17c88c)

Signed-off-by: Andrew Ross <andrross@amazon.com>

github-actions · 2023-06-22T16:39:10Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/18186/
CommitID: 0dfe96e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-22T17:21:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testPressureServiceStats

URL: https://build.ci.opensearch.org/job/gradle-check/18187/
CommitID: daeaa52
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <andrross@amazon.com>

This change was backported to 2.x in #7945. Signed-off-by: Andrew Ross <andrross@amazon.com>

…request (opensearch-project#7321) (opensearch-project#7945) * Compress and cache cluster state during validate join request (opensearch-project#7321) * Compress and cache cluster state during validate join request Signed-off-by: Aman Khare <amkhar@amazon.com> * Add changelog and license Signed-off-by: Aman Khare <amkhar@amazon.com> * Add javadoc and correct styling Signed-off-by: Aman Khare <amkhar@amazon.com> * Add new handler for sending compressed cluster state in validate join flow and refactor code Signed-off-by: Aman Khare <amkhar@amazon.com> * Refactor util method Signed-off-by: Aman Khare <amkhar@amazon.com> * optimize imports Signed-off-by: Aman Khare <amkhar@amazon.com> * Use cluster state version based cache instead of time based cache Signed-off-by: Aman Khare <amkhar@amazon.com> * style fix Signed-off-by: Aman Khare <amkhar@amazon.com> * fix styling 2 Signed-off-by: Aman Khare <amkhar@amazon.com> * Use concurrent hashmap instead of cache, add UT class for ClusterStateUtils Signed-off-by: Aman Khare <amkhar@amazon.com> * style fix Signed-off-by: Aman Khare <amkhar@amazon.com> * Use AtomicReference instead of ConcurrentHashMap Signed-off-by: Aman Khare <amkhar@amazon.com> * Use method overloading to simplify the caller code Signed-off-by: Aman Khare <amkhar@amazon.com> * Resolve conflicts Signed-off-by: Aman Khare <amkhar@amazon.com> * Change code structure to separate the flow for JoinHelper and PublicationTransportHelper Signed-off-by: Aman Khare <amkhar@amazon.com> * Remove unnecessary input.setVersion line Co-authored-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Aman Khare <85096200+amkhar@users.noreply.github.com> --------- Signed-off-by: Aman Khare <amkhar@amazon.com> Signed-off-by: Aman Khare <85096200+amkhar@users.noreply.github.com> Co-authored-by: Aman Khare <amkhar@amazon.com> Co-authored-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit b17c88c) * Change version guard to 2.9.0 Signed-off-by: Andrew Ross <andrross@amazon.com> --------- Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Aman Khare <85096200+amkhar@users.noreply.github.com>

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <andrross@amazon.com>

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

andrross requested review from Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, dblock, dbwiddis, dreamer-89, gbbafna, kartg, kotwanikunal, mch2, nknize, owaiskazi19, reta, ryanbogan, saratvemulapalli, setiah, shwetathareja and tlfeng as code owners June 6, 2023 22:59

andrross mentioned this pull request Jun 6, 2023

TokenManager Interface #7452

Merged

6 tasks

andrross force-pushed the backport/backport-7321-to-2.x branch from bcde254 to 91f9db9 Compare June 8, 2023 18:02

shwetathareja reviewed Jun 14, 2023

View reviewed changes

server/src/main/java/org/opensearch/cluster/coordination/JoinHelper.java Show resolved Hide resolved

andrross force-pushed the backport/backport-7321-to-2.x branch from 91f9db9 to c785e9f Compare June 15, 2023 21:52

andrross requested a review from sachinpkale as a code owner June 15, 2023 21:52

andrross force-pushed the backport/backport-7321-to-2.x branch from c785e9f to fb56099 Compare June 16, 2023 23:33

andrross added a commit to andrross/OpenSearch that referenced this pull request Jun 20, 2023

Update compressed cluster state version guard

0f16817

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <andrross@amazon.com>

andrross mentioned this pull request Jun 20, 2023

Update compressed cluster state version guard #8185

Merged

6 tasks

kotwanikunal approved these changes Jun 20, 2023

View reviewed changes

andrross force-pushed the backport/backport-7321-to-2.x branch from fb56099 to 0dfe96e Compare June 21, 2023 15:27

amkhar and others added 2 commits June 22, 2023 09:36

Change version guard to 2.9.0

daeaa52

Signed-off-by: Andrew Ross <andrross@amazon.com>

andrross force-pushed the backport/backport-7321-to-2.x branch from 0dfe96e to daeaa52 Compare June 22, 2023 16:37

andrross merged commit 4b3b30d into opensearch-project:2.x Jun 22, 2023

andrross added a commit to andrross/OpenSearch that referenced this pull request Jun 22, 2023

Update compressed cluster state version guard

3daddc6

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <andrross@amazon.com>

kotwanikunal pushed a commit that referenced this pull request Jun 22, 2023

Update compressed cluster state version guard (#8185)

4be8918

This change was backported to 2.x in #7945. Signed-off-by: Andrew Ross <andrross@amazon.com>

andrross deleted the backport/backport-7321-to-2.x branch February 8, 2024 19:47

Conversation

andrross commented Jun 6, 2023

Uh oh!

github-actions bot commented Jun 6, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 6, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

codecov bot commented Jun 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Jun 8, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

kotwanikunal commented Jun 13, 2023

Uh oh!

Uh oh!

github-actions bot commented Jun 15, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 16, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 17, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 21, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 21, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

andrross commented Jun 22, 2023

Uh oh!

github-actions bot commented Jun 22, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 22, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Jun 6, 2023 •

edited

Loading